In the latter half of the twentieth century, there began a phenomenon known as the information revolution. While the information revolution is a historical development broader in scope than any one event or machine, no single device has come to represent the information revolution more than the digital electronic computer. The development of computer systems has surely been a revolution. Each year, computer systems grow faster, store more data, and provide more applications to their users.
A modern computer system typically comprises hardware in the form of one or more central processing units (CPU) for processing instructions, memory for storing instructions and other data, and other supporting hardware necessary to transfer information, communicate with the external world, and so forth. From the standpoint of the computer's hardware, most systems operate in fundamentally the same manner. Processors are capable of performing a limited set of very simple operations, such as arithmetic, logical comparisons, and movement of data from one location to another. But each operation is performed very quickly. Programs which direct a computer to perform massive numbers of these simple operations give the illusion that the computer is doing something sophisticated. What is perceived by the user as a new or improved capability of a computer system is made possible by performing essentially the same set of very simple operations, but doing it much faster. Therefore continuing improvements to computer systems require that these systems be made ever faster.
The overall speed at which a computer system performs day-to-day tasks (also called “throughput”) can be increased by making various improvements to the computer's hardware design, which in one way or another increase the average number of simple operations performed per unit of time. The overall speed of the system can also be increased by making algorithmic improvements to the system design, and particularly, to the design of software executing on the system. Unlike most hardware improvements, many algorithmic improvements to software increase the throughput not by increasing the average number of operations executed per unit time, but by reducing the total number of operations which must be executed to perform a given task.
Complex systems may be used to support a variety of applications, but one common use is the maintenance of large databases, from which information may be obtained. Large databases usually support some form of database query for obtaining information which is extracted from selected database fields and records. Such queries can consume significant system resources, particularly processor resources, and the speed at which queries are performed can have a substantial influence on the overall system throughput.
Conceptually, a database may be viewed as one or more tables of information, each table having a large number of entries (analogous to rows of a table), each entry having multiple respective data fields (analogous to columns of the table). The function of a database query is to find all rows, for which the data in the columns of the row matches some set of parameters defined by the query. A query may be as simple as matching a single column field to a specified value, but is often far more complex, involving multiple field values and logical conditions. A query may also involve multiple tables (referred to as a “join” query), in which the query finds all sets of N rows, one row from each respective one of N tables joined by the query, where the data from the columns of the N rows matches some set of query parameters.
Execution of a query involves retrieving and examining records in the database according to some search strategy. For any given logical query, not all search strategies are equal. Various factors may affect the choice of optimum search strategy and the time or resources required to execute the strategy.
For example, one of the factors affecting query execution is the sequential order in which multiple conditions joined by a logical operator, such as AND or OR, are evaluated. The sequential order of evaluation is significant because the first evaluated condition is evaluated with respect to all the entries in a database table, but a later evaluated condition need only be evaluated with respect to some subset of records which were not eliminated from the determination earlier. Therefore, as a general rule, it is desirable to evaluate those conditions which are most selective first. Query execution may be affected by other and additional factors.
In particular, query execution can be significantly affected by the use of certain auxiliary database structures. An auxiliary database structure is a structured collection of information derived from one or more tables of the database, which may be used to more efficiently execute a database query. One well known type of auxiliary database structure is an index. An index is conceptually a sorting of entries in a database table according to the value of one or more corresponding fields (columns). If a query includes an indexed value as a condition, it may be advantageous to use the index to determine responsive records, rather than examine each record in the applicable table.
Auxiliary database structures, such as indexes, are typically defined by a database designer, administrator or similar person. A well-designed database typically contains various auxiliary database structures to support query execution or for other purposes. Once defined, these structures are automatically maintained by database management software as changes are made to the underlying database records.
To support database queries, large databases typically include a query engine which executes the queries according to some automatically selected search (execution) strategy, using the known characteristics of the database and other factors. Some large database applications further have query optimizers which construct search strategies, and save the query and its corresponding search strategy for reuse. In such systems, it may be possible to construct and save multiple different query execution strategies for a single query.
Where an auxiliary database structure, such as an index, exists and is useful in executing a query, a query optimizer may construct an execution strategy to take advantage of the existing auxiliary database structure as a shortcut to executing the query. However, in many complex queries, it is desirable or essential to employ some auxiliary database structure in executing the query, but no such defined auxiliary database structure already exists. The optimizer will generally choose an alternative execution strategy which does not require the auxiliary database structure, and which may consume significantly more resource. In these cases, the query optimizer may generate an execution strategy which itself constructs the needed auxiliary database structure as a temporary object for use in executing the query, but this also involves further expenditure of resource.
While there are obvious benefits to maintaining auxiliary database structures, the maintenance of such structures is not free. Every auxiliary database structure which is defined by the database designer for supporting queries and the like must be maintained concurrently with the data in the database tables, i.e., as updates to database table records are made, the applicable auxiliary database structure or structures must likewise be updated so that the data contained therein agrees with the underlying data in the database tables, from which it was derived. Updating numerous auxiliary database structures is a significant overhead cost of maintaining a database. As more auxiliary database structures are added to the database definition, this overhead burden increases. For these reasons, an auxiliary database structure should not be defined to the database unless the resource savings from query execution or other functions which use the auxiliary database structure is sufficiently large to justify the overhead cost of maintaining the auxiliary database structure.
If a database designer, administrator or similar person is presented with a hypothetical or actual auxiliary database structure for inclusion in a database definition, it is difficult to predict the effect of the presence of such an auxiliary database structure on query execution performance, and therefore difficult to determine whether the overhead burden of the auxiliary database structure will be justified. Although not necessarily recognized in the art, a need exists for improved techniques for determining auxiliary database structures for inclusion in the definition of a database, an in particular, a need exists for techniques for projecting the effect of the presence of an auxiliary database structure on query execution performance.